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1.0  INTRODUCTION 


As  part  of  the  Operational  Decision  Aids  (ODA)  program,  Integrated 
Sciences  Corporation  is  conducting  experimental  investigations  of  man- 
machirie  interface  (ill'll)  concepts.  The  current  task  is  directed  toward  the 
development  and  evaluation  of  Continuous  Subjective  Functions  (CSF)  as  they 
apply  to  computer-aided  decision  making.  The  use  of  Continuous  Subjective 
Functions  is  one  means  by  which  a decision  maker  (e.g..  Task  Force  Commander 
(TFC))  can  convey  his  estimate  of  a given  situation  to  the  decision  aiding 
hardware.  As  such,  it  can  represent  an  alternative  to  less  comprehensive 
methods,  such  as  eliciting  scalar  estimates  for  man- to-machi ne  transfer  of 
subjective  judgments. 

One  way  to  implement  a CSF  via  decision  aiding  equipment  is  for 
the  decision  maker  to  "draw"  the  curve,  line,  or  surface  representing  his 
estimate  on  a computer-driven  graphics  display.  Various  types  of  control 
hardware  are  available  to  perform  drawing  tasks  in  conjunction  with  an 
interactive  graphics  display;  among  them  are  the  light  pen,  the  track  ball, 
and  the  joy  stick.  The  light  pen  is  one  of  the  most  commonly  used  devices; 
it  consists  of  a hand-held  fiber-optic  tube  that  can  be  "pointed"  at  a 
light  source  (e.g.,  cursor)  on  the  screen  and  used  to  move  the  cursor  to 
produce  a line.  The  joy  stick  and  track  ball  are  both  analog  devices.  The 
joy  stick  resembles  a pilot's  control  stick;  the  track  ball  consists  of  a 
recess-mounted  sphere  that  can  be  rotated  by  the  palm  of  the  hand.  Both 
are  equipped  with  potent i ometers , the  signals  of  which  are  converted  to 
digital  x-  and  y-values,  so  that  the  cursor  on  the  screen  moves  in  direct 
proportion  to  the  movement  of  the  device. 

This  pilot  study  was  undertaken  to  evaluate  the  light  pen,  track 
ball,  and  joy  stick  in  relation  to  types  of  drawing  tasks  to  be  performed  in 
subsequent  CSF  research.  By  determining  in  advance  which  of  the  three  control 
devices  yields  the  best  performance  in  CSF-type  tasks  helps  to  avoid  con- 
founding the  main  experiment.  Accordingly,  the  primary  experimental  null 


hypothesis  was  that  there  is  no  significant  difference  among 
report  documents  the  results  of  the  three-factor  analysis  of 
describes  the  results  of  tests  on  observed  differences  among 
develops  a criterion  function  that  aids  the  selection  of  the 


devices.  This 
variance, 
devices,  and 
best  device. 
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2.0  EXPERIMENTAL  DESIGN 


2.1  OPERATOR  TASKS 

The  geneiation  of  a Continuous  Subjective  Function  (CSF)  may 
require  a variety  of  "drawing"  tasks  from  the  operator.  Thus  the  graphic 
representation  of  a subjective  function  may  be  accomplished  by  drawing 
straight  line  segments,  curves,  or  surfaces,  depending  on  the  function. 
Furthermore,  curves  may  either  be  continuous  (e.g.,  circles)  or  discrete 
(e.g.,  histograms).  In  addition,  a drawing  task  may  be  further  classified 
into  three  categories: 

• Sketching 

• Tracing 

• Pseudo-tracing 

In  this  context,  "sketching"  refers  to  the  transfer  of  a mental  image  to 
the  graphics  display  screen  without  the  aid  of  visual  reference--or  anchoi — 
points.  "Tracing"  involves  generating  a visual  image  aided  by  a series  of 
closely  spaced  anchor  points;  when  done  on  a CRT,  tracing  is  analogous  to 
a "connect- the- dots"  routine.  "Pseudo-tracing"  involves  aspects  of  both 
sketching  and  tracing:  there  may  be  certain  points,  lines,  or  other  data 

available  as  visual  references,  but  they  are  not  so  many  that  the  task  be- 
comes simply  mechanical. 

Since  sketching  and  pseudo-traci ng  of  straight  lines  and  curves 
are  the  types  of  drawing  tasks  involved  in  the  production  of  a CSF,  the 
experiment  included  both  types.  The  experimental  tasks  excluded  (insofar 
as  possible)  a subject  operator's  artistic  ability  or  mathematical  sophis- 
tication as  an  influence  on  the  results.  Another  factor  influencing  the 
selection  of  experimental  tasks  was  the  need  for  straightforward  measure- 
ment of  operator  performance  across  control  devices.  Figure  1,  for  instance, 
shows  two  possible  operator  estimates  (broken  lines)  of  a CSF  (solid  line). 
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CSF 

Operotor|  Estimate 

Operator^  Estimate 


Figure  1.  Example  CSF  Estimates. 


To  measure  which  operator  estimate  is  a better  rendition  of  the  desired 
shape  was  considered  needlessly  complex  for  this  experiment.  Instead,  one 
or  two  simple  geometric  figures  were  needed,  which  (l)  required  the  types 
of  drawing  tasks  desired,  (2)  were  commonly  understood,  and  (3)  allowed 
direct  measurement  of  deviations.  Accordingly,  two  commonly  known  geo- 
metric figures  were  chosen  for  the  drawing  task:  the  equilateral  triangle 

and  the  circle.  Doth  figures  readily  yield  commonly  held  and  repeatable 
mental  images;  this  helped  to  measure  more  directly  performance  due  to  the 
control  devices.  Selection  of  these  two  figures  also  allowed  the  definition 
of  the  dependent  variables  to  be  relatively  straightforward  (see  Section 
2.2)  . 


Finally,  drawing  the  two  figures  involved  both  sketching  and 
pseudo-traci ng  tasks.  Drawing  the  circle  is  defined  as  purely  a sketching 
task:  no  visual  anchor  points  were  available  to  the  subject  operator. 
Drawing  the  equilateral  triangle  as  a whole  is  taken  to  be  a sketching  task; 
but  additional  distinctions  were  made  for  generating  the  individual  line 
segments  of  the  triangle.  These  distinctions  are  as  follows:  first,  the 

order  and  direction  in  which  the  sides  of  the  triangle  were  to  be  drawn 
were  specified,  in  order  to  simplify  the  sampling  of  points  (done  in  soft- 
ware) on  the  line  segments.  These  specifications  are  shown  in  Figure  2. 


- 1,  - 


V 

START 


1st  side 


Figure  2.  Triangle  Sketching  Procedure. 

Drawing  the  first  side  was  defined  as  a pseudo-sketching  task;  since  the 
line  was  to  be  horizontal,  the  operator  could  take  advantage  of  the  edge 
of  J;he  screen  as  a reference.  The  second  side  required  sketching:  the 

operator  had  to  judge  the  proper  length  and  slope  of  the  line  without  the 
benefit  of  easily  used  landmarks.  Drawing  the  third  side  consisted  of  a 
pseudo-sketching  task  with  anchor  points,  i .e. , two  vertices  of  the  triangle 
the  operator  had  only  to  draw  a straight  line  segment  between  the  two  points 

In  summary,  drawing  the  circle  was  a sketching  task  only,  while 
drawing  the  triangle  involved  a transition  from  pseudo-tracing  to  sketching 
to  pseudo-tracing  wi th  anchor  points.  These  distinctions  helped  to  define 
the  experimental  dependent  variables  which  are  the  performance  measures  of 
the  circle-  and  triangle-drawing  tasks. 

2.2  DEPENDENT  VARIABLES 

To  evaluate  performance  of  three  control  devices  for  these 
drawing  tasks,  nine  performance  variables  were  defined.  Eight  were 
recorded  for  each  operator-drawn  triangle  and  one  for  each  circle.  Thus, 
both  straight  and  curved  line  sketching  were  measured,  as  well  as  straight 
line  pseudo-tracing. 


First,  the  straightness  of  each  side  of  the  triangle  v/as 
measured.  For  each  side,  points  were  sampled  from  the  drawn  line  according 
to  a sol tware-control led  time/distance  criterion.  Linear  regression 
coefficients  for  a least-squares  fit  were  calculated  (Equations  1 and  2) 
from  the  x-  and  y-coord i nates  of  the  sampled  points  on  the  line: 


al 


Ex  ■ y ; 


Ex: 


ExiEy; 

n 

(Ex,)2 

n 


(i) 


n n 

where  the  summations  over  i go  from  1 to  n,  the  number  of  sampled  points. 
Next,  estimated  valugs  of  y.  were  computed  for  each  value  of  x . on  the 
line  according  to  the  regression  equation  (3): 

Vi  = alxi  + a2  (3) 

The  residual  is  then  defined  to  be: 

Ri  = (yi  -yi)2  (*») 

where  y.  equals  the  raw  y-value  of  the  sampled  point  on  the  drawn  line. 

Then  for  each  of  the  three  sides,  the  mean  residual  and  standard  deviation 
of  the  residuals  were  calculated  according  to  Equations  5 and  6 respectively 


R.= 

i 


ERj 

n-2 


oR  = Ar2  - (ER. ) 2/n 


(n-  1 ) 


(5) 

(6) 


To  measure  how  closely  the  triangle  drawn  on  the  screen  resembled 
an  equilateral  triangle,  standard  deviations  for  the  length  of  the  sides 
and  for  the  size  of  the  angles  were  computed.  The  lengths,  L ; , of  the  three 
sides  were  calculated  according  to  the  Euclidian  distance  between  the 
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respective  vertices.  Mean  length,  L,  was  computed;  the  standard  deviation 
for  the  lengths  wa s obtained  by: 

°L  = ' 3([)2  (7) 

The  angles,  ».  , were  determined  (again  from  the  vertices  of  the  triangle) 

and  the  standard  deviation,  a , was  comDuted  by: 

o ' 

oa  = Aa.2  - 3 ( 60)7  (8) 

2 

The  ingle  performance  variable  recorded  for  the  circle  was  the 
standard  deviation  of  di stance  (rad ius) , r j , from  the  centroid  of  the 
sampled  points  of  the  drawn  figure.  The  centroid  (xr,  y_)  was  taken  to  be 
the  mean  of  the  x-  aQd  y-coordi nates  of  the  sampled  points.  Distance  from 
the  centroid  to  each  sampled  point  was  determined  by  the  Euclidian  distance 
formula,  so  that  the  standard  deviation  was  given  by: 

or  = - Ju}  - n(P)2  (9) 


To  summarize,  nine  performance  measures  were  recorded  for  each 

trial: 

(1-3)  Three  values  of  , the  standard  deviation  of 
residuals  for  each'side  of  the  triangle. 

(l|-6)  Three  values  of  Rj,  the  mean  residuals  for 
each  side  of  the  triangle. 

(7)  One  value  of  a^,  the  standard  deviation  of 
the  lengths  of  the  sides  of  the  triangle. 

(8)  One  value  of  o , the  standard  deviation  of 

Ct 

the  angles  of  the  triangles. 

(9)  One  value  of  a , the  standard  deviation  of 
the  radii  of  tfie  circle. 


2.3  INDEPENDENT  VARIABLES 


The  three  independent  variables  for  the  experiment  v/e re: 

(1)  Subject  operators 

(2)  Interactive  graphics  control  devices 

(3)  Repl icati ons 

Five  subject  operators  were  used.  All  were  U.C.L.A.  undergraduates  in  the 
fields  of  engineering,  computer  science,  and  psychology.  Three  control 
devices  were  used:  joystick,  trackball,  and  lighlpen.  Four  complete 

replications  were  performed;  each  replication  consisted  of  four  triangles 
and  four  circles  drawn  per  each  control  device  per  subject. 

■V. 

2. A ANOVA  DESIGN 


The  effects  of  possible  differences  in  devices,  operators,  and 
replications  on  performance  were  of  interest.  Therefore  it  was  decided 
to  conduct  a three  factor,  fully  replicated,  randomized  block  factorial 
experiment  to  determine  if  there  were  significant  performance  differences 
due  to  the  factors  or  their  interactions.  There  were  three  devices,  five 
operators,  and  four  replications.  Each  replication  on  a performance  measure 
consisted  of  four  data  points  generated  by  an  operator  during  a single 
session.  Thus,  the  data  matrix  for  a single  performance  measure  and  one 
replication  has  the  structure  of  Table  1.  The  full  data  matrix  for  a 
performance  is  3 devices  x 5 operators  x^  replications  x^  data  points 
per  replication.  The  model  is: 


Y...  = U + R.  + D.  + RD.  . + 0,  + R0..  + 0D..  + RDO. .. 

ijkm  i j ij  k ik  jk  ijk 

+ Em(ijk) 


(10) 


The  levels  of  devices  are  clearly  fixed  and  not  chosen  at  random. 
Operators  were  not  chosen  at  random  from  any  population  of  potential  users 
of  the  control  devices  in  a military  setting.  Therefore  the  results  of 


the  experiment  are  not  general i zable  to  any  population  and  "operators" 
must  be  considered  a fixed  variable.  Replications  are  assumed  to  be  a 
random  variable.  Since  there  are  two  factors  at  fixed  levels  and  one  at 
random  levels,  the  model  is  a mixed  effects  model. 


/ 
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3.0 


EXPERIMENTAL  PROCEDURE 


3.1  SINGLE  TRIAL  PROCEDURE 

Each  experimental  session  was  conducted  by  a test  director  and 
two  subject  operators.  For  each  trial,  the  test  director  selected  the 
control  device  to  be  used  for  that  trial;  a single  trial  consisted  of 
drawing  one  triangle  and  one  circle  with  the  selected  control  device. 

At  the  beginning  of  a session,  the  test  director  scheduled,  via 
the  teletype,  the  software  package  for  the  experiment.  Responding  to  an 
interactive  query  at  the  teletype,  the  test  director  entered  the  identifi- 
cation number  assigned  to  whichever  subject  operator  was  executing  the 
trial.  The  test  director  then  entered  a number  designating  the  control 
device  to  be  used  for  that  trial.  The  test  director  randomized  his 
selection  of  control  devices  to  minimize  any  sequential  effects  across 
devices.  In  general  a subject  operator  did  not  use  the  same  control  de- 
vice for  two  consecutive  trials. 

Once  the  program  was  scheduled  and  a control  device  selected, 
the  subject  operatoi — seated  at  the  display  console — began  the  trial.  On 
the  screen  appeared  a frame  marking  the  boundaries  of  the  screen  area 
where  the  figures  were  to  be  drawn,  and  the  legend  "Position  Cursor."  The 
subject  operator  used  the  designated  control  device  to  move  the  cursor  to 
the  screen  position  he  chose  as  the  starting  point  for  the  first  leg  of 
the  equilateral  triangle.  Once  he  had  the  cursor  positioned  at  the  chosen 
point,  the  operator  depressed  the  lit  function  button,  thus  (1)  marking  the 
coordinates  of  the  start  point  for  the  first  line  segment,  and  (2)  freeing 
the  control  device  to  "draw"  the  first  leg  of  the  triangle.  At  the  end  of 
the  first  leg,  the  operator  depressed  the  function  button  again  to  mark  the 
coordinates  of  the  second  vertex  of  the  triangle.  The  second  leg  was  drawn, 
and  the  function  button  depressed  to  mark  the  starting  point  (vertex)  of 
the  third  leg.  The  operator  drew  the  final  leg;  when  this  third  line  seg- 
ment was  within  a software-specified  distance  of  the  initial  start  point 


for  the  triangle,  the  figure  was  automatically  closed.  For  sampling 
purposes,  each  side  of  the  triangle  had  to  be  at  least  two  inches  long; 
if  the  completed  operator-drawn  figure  did  not  satisfy  this  requirement, 
a message  appeared  on  the  display  screen  instructing  the  operator  to  re- 
draw the.  figure.  In  this  case  the  operator  repositioned  the  cursor  and 
proceeded  as  before. 

If  the  triangle  met  the  specifications,  the  operator  next  pro- 
ceeded to  draw  the  circle.  Using  the  same  control  device  designated  at 
the  start  of  the  trial,  he  positioned  the  cursor  at  the  desired  start 
point,  again  within  the^frame  appearing  on  the  display.  To  draw  the 
figure,  the  operator  depressed  the  single,  lit  function  button  and  pro- 
ceeded to  use  the  control  device  to  produce  the  circle.  In  this  case, 
there  was  no  requirement  for  direction  in  which  the  circle  was  drawn: 
the  operator  was  free  to  proceed  either  clockwise  or  counterclockwise. 
Again,  the  figure  was  closed  automatically  when  the  cursor  was  within  the 
prespecified  distance  of  the  initial  point.  If  the  completed  circle  was 
less  than  the  specified  one-inch  radius,  the  operator  was  i nstructed--via 
a message  on  the  display--to  redraw  the  figure.  If  the  circle  satisfied 
this  requirement,  the  trial  was  over,  and  the  test  director  defined  the 
next  trial  by  selecting  another  of  the  control  devices  and/or  the  other 
member  of  the  subject  operator  team. 

3.2  SCHEDULE 

Prior  to  the  experimental  sessions  performed  for  data  collection, 
the  subject  operators  participated  in  a series  of  training  sessions  under- 
taken to  familiarize  the  subjects  with  the  hardware  and  experimental  pro- 
cedures and  to  help  minimize  learning  effects  during  the  data  collection 
phase.  The  training  sessions  were  also  used  to  identify  and  discourage 
any  operating  strategies  developed  by  a subject  that  could  affect  the 
experimental  results.  The  test  director  was  on  hand  to  educate  the  sub- 
jects in  the  use  of  the  hardware,  monitor  practice  runs,  and  provide 
training  feedback. 
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Data  gathering  sessions  were  initiated  once  the  subjects  were 
familiar  with  equipment  and  procedures.  A single  session  consisted  of 
the  test  director  and  two  subject  operators  and  was  scheduled  for  two 
hours.  The  subjects  performed  (rials  according  to  the  sequence  dictated 
by  the  test  director.  Since  the  individual  trials  could  be  completed 
fairly  quickly,  each  subject  was  able  to  complete  one  replication  per 
session;  any  time  remaining  in  the  two-hour  session  was  devoted  to 


training  tasks  pertaining  lo  the  main  experiment  (in  progress). 


4.0  DISCUSSION  OF  RESULTS 


*♦.1  TESTS  OF  SIGNIFICANCE 


Tests  of  significance  were  performed 
for  the  first-  and  second-order  interactions, 
ratios  for  each  per formance  measure  are  given 
significance  are  shown  in  Table  2. 


for  each  main  effect  and 
Computed  values  of  the  F 
in  Appendix  A;  levels  of 


Several  preliminary  conclusions  were  indicated  by  the  results 
shown  in  Table  2.  First,  devices  were  clearly  significant  for  all 
performance  measures  involving  straight-  or  curved-line  drawing  tasks. 
Second,  replications  appeared  to  be  significant.  This  was  discounted 
however,  because  the  Replication  x Device  interaction  was  not  significant, 
i.e.,  there  were  no  significant  learning  or  boredom  effects  that  selec- 
tively affected  devices.  Third,  operators  appeared  to  be  significant  in 
tasks  involving  straight  lines.  In  this  case,  the  results  could  not  be 
overlooked  since  Device  x Operator  interaction  was  also  significant  for 
six  of  the  eight  performance  measures  involving  straight  line  drawing. 


The  Device  x Operator  interactions  were  plotted  for  the  signifi- 
cant performance  measures  (Figures  3“8) . Examination  of  the  interactions 
for  performance  measures  on  the  second  leg  of  the  triangle  (Figures  3 and 
A)  shows  drastic  differences  among  operators  for  both  the  joy  stick  and 
light  pen.  These  interactions  also  show  that  the  common  element  across 
all  operators  is  that  the  track  ball  yields  the  best  (or  insignificantly 
different  from  the  best)  performance.  Figures  5 and  6,  showing  the  inter- 
action for  performance  measures  on  the  third  leg  of  the  triangle,  are  less 
clear  cut.  It  appears  that  the  results  are  highly  individualized  by 
subject  operators.  Clearly,  operators  2 and  3 differ  markedly  in  perfor- 
mance from  the  other  members  of  the  experimental  group.  (In  a real  world 
situation,  these  two  might  be  assigned  to  more  appropriate  jobs.)  Other- 
wise, among  operators  1,  4,  and  5,  the  track  ball  again  seems  to  yield 
the  best  (or  insignificantly  different  from  the  best)  performance.  Figures 


Table  2.  Results  of  ANOVA  Significance  Tests. 


Figure  5-  Dev i ce x Operator  Interaction  for  Standard  Deviation 
of  Residuals,  Triangle  Side  Three  (o_  ). 
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7 and  8,  for  measures  of  the  regu 1 ar i ty  of  the  triangle  (u.  and  a)  show 
that — except  for  one  operator  — there  appears  to  be  little  difference  among 
the  devices.  This  might  be  expected  because  devices  alone  were  not  signi- 
ficantly different  for  these  performance  measures  (Table  2)  and  because 
producing  an  equilateral  triangle  can  be  expected  to  depend  more  on  an 
operator's  spatial  judgement  than  on  the  control  device  used. 


A. 2 RANK  OF  DEVICES 

Since  control  devices  were  significant  for  straight  and  curved 
line  drawing  tasks,  the  total  means  for  devices  were  ranked  numerically 
and  the  contrasts  among  devices  tested  by  Scheffe's  method  to  provide  a 
preliminary  indication  of  which  device  was  best.  Results  are  shown  in 
Table  3*  From  the  table,  it  can  be  seen  that  on  the  basis  of  observed 
differences  among  the  devices,  the  track  ball  was  most  often  the  superior 
device.  In  conjunction  with  the  results  of  Scheffe's  test,  the  track  ball 
appears  to  be  the  best  device  for  straight-line  tracing  and  Rj)  and 

is  clearly  the  best  for  straight-line  sketching  (or0  and  f^,).  Judging 
from  observed  differences  only,  the  light  pen  yielded  the  best  performance 
for  straight-line  pseudo-tracing  with  anchor  points  (o^  and  R^) . Tests 
on  the  means  of  these  measures,  however,  were  inconclusive.  In  the  first 
case  (op^),  the  contrast  between  the  light  pen  and  joy  stick  is  signifi- 
cant, but  the  track  ball  is  not  significantly  different  from  either.  In 
the  second  case  (R3)  no  mean  is  significantly  different  from  any  other 
mean.  For  these,  two  performance  measures,  then,  it  is  difficult  to  judge 
between  the  track  ball  and  light  pen  as  to  which  is  the  better  device  for 
this  pseudo-tracing  task.  For  the  performance  measure  (a  ) for  curved- 
line  sketching,  Scheffe's  test  did  not  support  the  observed  numerical 
superiority  of  the  track  ball.  In  this  case,  as  for  R3,  none  of  the  means 
was  significantly  different  from  any  other  mean. 


Scheffe's  test  is,  of  course,  less  sensitive  to  observed  differ- 
ences than  are  other  tests  on  means,  hence  one  possible  source  of  inconclusive 
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results  for  three  of  the  performance  measures.  The  information  displayed 
in  Table  3,  however,  suggests  that  the  track  ball  is  a reasonable  choice 
among  the  devices  for  both  straight-  and  curved-line  drawing  tasks.  The 
track  ball  is  clearly  best  for  straight-line  sketching  and  also  displays 
the  best  performance  value  (lowest  total  mean)  for  curved-line  sketching 
(or) . 


*1.3  CP  I TER  I ON  FUNCTION  EVALUATION 


A second  evaluation  of  the  control  devices  was  carried  out  by 
means  of  a normalized,  weighted,  linear  sum  criterion  function.  The  first 
step  in  formulating  the  criterion  function  was  to  simplify  the  evaluation 
by  eliminating  irrelevant  or  redundant  performance  measures  from  consider- 
ation. Therefore,  and  au  (measures  of  regularity  of  the  triangle)  were 
eliminated  because  devices  were  not  significantly  different  for  these 
variables  (Table  2).  In  addition,  the  three  standard  deviations  of 
residuals  of  the  sides  of  the  triangles  (ctr^  , at^ » oR^)  were  eliminated 
because--for  purposes  of  the  criterion  f unct ion-- they  were  considered 
redundant  with  the  three  mean  residuals  of  the  sides  of  the  triangle 
(R| , R2,  and  R3).  Of  the  four  remaining  performance  measures  of  interest, 
three  were  measures  of  straight-line  drawing  tasks  and  one  was  a measure 
of  curved- line  drawing.  They  are: 

(1)  R|  : a measure  of  straight-line  pseudo-t rac i ng  with 

a reference  frame 

(2)  R^  : a measure  of  straight-line  sketching 

(3)  R3  : a measure  of  stra ight- 1 i nc  pseudo-tracing  with 

anchor  points 

(*i)  ar  : a measure  of  curved-line  sketching. 


The  functional  form  of  the  criterion  function  is  given  by  Equation  H: 


PM 


CFi  -*  “j 
j=*l 
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PM* 
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(11) 
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where : 


and : 


CFj  = criterion  function  value  of  the  i1*1  device 

wj  = the  weighting  coefficient  of  the  jth  performance 
measure 

PMjj  = the  experimental  value  of  the  performance  measure 
of  the  i^h  device  on  the  j performance  measure 
(total  mean  experimental  value) 

PMv  = the  value  of  the  best  (minimum)  performance  measure 
among  the  three  devices  for  the  j*-*1  performance 
measure 


U 

1 w.  = 1 .0  (12) 

j = l J 


Since  three  of  the  performance  measures  pertained  to  straight-line 
drawing  tasks,  it  was  decided  to  use  w0  as  the  sensitivity  variable,  so 
that  f rom  (12)  : 


% 


+ WFU 


+ v/R3  = 1 " Wc 


03) 


The  order  of  importance  of  the  types  of  straight-line  drawing  tasks  was 
defined  and  subjective  ratio  factors  assigned.  Since  sketching  was 
considered  most  consistent  with  CSF-type  tasks,  R2  was  assigned  the  rank 
of  one;  R|  and  R3  were  ranked  two  and  three,  respectively.  The  relative 
importance  of  the  straight-line  drawing  tasks  is  expressed  in  Equations 
(1*0  and  (15)  by  subjective  constant  ratio  constraints. 


.20 

.25 
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Wp-  .15 
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(15) 
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The  sensitivity  analysis  consisted  of  selecting  a value  for  the 
sensitivity  variable,  w0  ; solving  the  system  of  equations  (13,  l**,  and 
15)  for  WRj  , wp2  and  w-^  ; and  using  these  solved  values  of  v/j  to  compute 
the  value  of  C F j (Equation  11). 

The  results  are  plotted  in  Figure  9 for  values  of  interest  of  the 
sensitivity  variable,  namely  from  0.4  to  0.8.  The  plot  indicates  that  the 
track,  ball  and  joy  stick  are  markedly  consistent  across  performance 
measures,  i.e.,  their  respective  performances  are  quite  similar  for  the 
types  of  drawing  tasks  (straight  vs.  curved  lines)  examined  here.  The 
slope  of  the  plot  for  the  light  pen  suggests  that  the  light  pen's 
performance  varies  among  the  drawing  tasks,  with  a relative  superiority 
for  curved  line  sketching.  The  primary  interpretation  of  the  criterion 
function  evaluation,  however,  is  that  the  track  ball  appears  to  be  always 
the  best  device  regardless  of  the  subjective  weighting  of  the  curved 
(w(0r))  line  sketching  performance  factor. 

CONCLUSIONS 

The  analysis  of  variance  indicated  that  the  three  interactive 
graphics  control  devices  examined  in  this  study  were  significantly  different 
for  a range  of  drawing  tasks — especial ly  straight-  and  curved-line 
sketching  of  the  type  used  to  represent  continuous  subjective  functions. 
Although  the  results  of  using  Scheffers  test  did  not  show  the  track  ball 
to  be  superior  in  all  cases,  they  provide  a reasonable  basis  for  selecting 
the  track  ball  as  the  preferred  device.  On  the  basis  of  the  criterion 
function  evaluation,  the  track  ball  is  clearly  the  best  device  overall. 

The  combined  impact  of  these  results,  therefore,  indicates  that  the  track 
ball  will  be  the  single  control  device  of  choice  for  all  subsequent  contin- 
uous subjective  function  appl i cat  ions. 


Table  k . ANOVA  Table:  Standard  Deviation  of  Residuals  (oD  ),  Side 
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Table  5.  ANOVA  Table:  Standard  Deviation  of  Residuals  (c.,  ),  Side 
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Table  3.  ANOVA  Table: 
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Table  9.  ANOVA  Table:  Mean  Residual  (R,) , Side 


ANOVA  Table:  Standard  Deviation  (a.)  for  Lengths  of  Side 


ANOVA  Table:  Standard  Deviation  (c  ) for  Angles  of  Triangle 
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